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FOREWORD 


The  Howitzer  Improvement  Program  (HIP)  self-propelled  M109A6 
howitzer  is  equipped  with  an  Embedded  Trainer  (ET) .  However,  the 
ET  was  not  included  in  the  HIP  Initial  Operational  Test  and 
Evaluation  (lOTE)  and,  consequently,  the  merits  of  the  system 
were  unknown  as  the  Army  prepared  for  the  pre-Army  Systems  Acqui¬ 
sition  Review  Council  (pre-ASARC)  meeting.  To  remedy  this  circ¬ 
umstance,  the  HIP  Product  Manager  (PM)  requested  support  from  the 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences^^ 

This  report  describes  the  research  undertaken  to  evaluate 
the  HIP  ET.  A  full-scale  assessment  was  not  possible  because  of 
the  limited  availability  of  a  HIP  howitzer.  Both  subjective 
reactions  to  the  ET  and  objective  performance  were  measured. 
Additionally,  a  detailed  screen-by-screen  analysis  of  the  tuto¬ 
rial  was  performed.^ 

^  A  major  MANPRINT  concern  in  the  development  of  the  HIP  is 
whether  or  not  the  entire  targeted  user  population  can  perform 
the  chief-of-section  functions  adequately.  This  report  addresses 
that  issue  and  provides  recommendations  to  increase  the  effec¬ 
tiveness  and  viability  of  the  HIP  ET. 
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ABBREVIATED  ASSESSMENT  OF  EMBEDDED  TRAINING 
FOR  THE  HOWITZER  IMPROVEMENT  PROGRAM  M109A6  HOWITZER 


Introduction 


The  scheduled  development  cycle  for  the  Howitzer  Improvement 
Program  (HIP)  self-propelled  M109A6  howitzer  included  a  pre-Army 
Systems  Acquisition  Review  Council  (pre-ASARC)  meeting  in 
November  1989.  During  preparations  for  that  review,  it  became 
apparent  that  no  empirical  research  had  been  accomplished  to 
assess  the  merits  of  the  Embedded  Trainer  (ET)  included  in  the 
HIP.  Because  the  ET  software  wasn't  available  early  enough  to 
allow  sufficient  time  for  the  using  unit  to  learn  how  to  employ 
it,  it  was  not  included  in  the  HIP  Initial  Operational  Test  and 
Evaluation  (lOTE)  undertaken  during  June-July  1989.  Accordingly, 
the  HIP  Product  Manager  (PM)  requested  support  from  the  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
to  conduct  such  an  evaluation  (DA,  AMCPM-HIP-LO,  28  July  1989) . 

The  Embedded  Training  concept  is  relatively  new  to  U.S.  Army 
systems.  The  intent  is  to  provide  the  using  unit  an  effective 
means  of  maintaining  the  proficiency  of  its  operators  and  crews 
during  sustainment  training  while  using  the  actual  end-item 
hardware  (Finley,  Alderman,  Peckham,  &  Strasel,  1988).  The  ET 
for  the  HIP  howitzer  is  built  into  the  Automated  Fire  Control 
System  (AFCS)  computer  on  board  the  HIP.  By  being  able  to  switch 
rapidly  from  a  fully  operational  system  to  one  that  can  support 
a  range  of  training  functions,  the  on-board  AFCS  provides  a 
potentially  powerful  and  versatile  ET  capability. 

The  HIP  AFCS  includes  a  visual  display  and  a  total  of  26 
keys  and  switches  used  by  the  soldier  to  control  the  operation 
and  input  information.  The  visual  display  is  an  orange-colored 
electroluminescent  screen.  Both  upper-case  and  lower-case 
alphabetic  characters  can  be  depicted  in  either  normal  (black 
characters  on  an  orange  background)  or  reverse  video.  The  AFCS 
includes  three  switches  to  control  power  to  the  AFCS  itself  and 
to  the  gun  servos,  and  to  switch  between  the  training  and 
operational  modes.  There  are  five  "hard-function"  keys,  a 
numeric  keypad  for  data  entry,  four  cursor  control  arrow  keys, 
and  four  "soft-function"  keys  whose  function  varies  with  (and  is 
displayed  upon)  the  particular  screen  that  requires  their  use. 

The  entering  of  alphabetic  characters  is  accomplished  via  the 
combined  use  of  the  arrow  keys  and  the  enter  key  to  select 
characters  displayed  on  the  screen. 

The  HIP  ET  provides  four  levels  of  training.  There  are  two 
levels  of  tutorial  instruction  (Levels  I  and  II)  and  two  levels 
of  scenario-based  training  (Levels  III  and  IV) . 
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Level  I  is  a  brief  tutorial  intended  to  provide  initial 
familiarization  for  the  trainee  having  no  prior  experience  with 
the  HIP.  The  trainee  learns,  in  step-by-step  fashion,  the 
physical  location  and  elementary  function (s)  of  the  various  input 
keys  appearing  on  the  front  of  the  AFCS.  The  second.  Level  II, 
tutorial  provides  system  start-up  training.  This  level  is  much 
more  detailed.  It  addresses  each  of  the  various  menus  which  must 
be  completed  during  the  HIP  AFCS  initialization  and  start-up 
procedures  in  a  highly  structured,  step-by-step  fashion.  The  ET 
instructs  the  trainee  concerning  the  inputs  required,  and,  as 
with  the  Level  I  Tutorial,  the  trainee  receives  feedback 
regarding  the  correctness  of  each  selection  he  makes.  At  both 
levels,  following  each  incorrect  response,  the  tutorial  provides 
information  intended  to  clarify  the  nature  of  the  response  or 
selection  needed  from  the  trainee,  and  redirects  him  back  to  the 
screen  wherein  the  incorrect  input  or  selection  was  made. 

The  two  levels  of  scenario-based  training  are  intended  for 
individuals  who  are  already  familiar  with  the  AFCS.  They  are 
intended  to  sustain  performance  at  acceptable  levels.  Both 
provide  the  capability  to  download  operator  performance  data 
summaries  through  a  downloading  port  provided  on  the  AFCS.  Three 
scenarios  may  be  selected:  "move  heavy,"  "shoot  heavy,"  or 
"balanced."  Level  IV  differs  from  level  III  in  that,  if  an 
instructor  is  present,  the  destinations  and  target  locations  can 
be  altered  and  instructor-generated  faults  (e.g.,  failure  of  the 
hydraulics,  communications,  navigational,  and  electrical 
systems)  can  be  introduced. 

With  the  publication  of  AR  602-2  (Headquarters,  Department 
of  the  Army,  1987),  the  regulation  implementing  the  U.s.  Army's 
MANPRINT  program,  there  existed  the  requirement  to  carefully  and 
fully  consider  the  soldier-related  factors  which  could  affect 
overall  performance  of  systems  under  development.  Since  its 
inception,  an  operational  requirement  of  the  HIP  was  that  it  be 
capable  of  being  successfully  operated  by  those  throughout  the 
entire  range  of  capabilities  inherent  within  the  population  of 
soldiers  in  the  military  occupational  specialty  13B.  The 
doctrine  governing  the  tactical  employment  of  the  HIP  requires 
frequent  and  rapidly-executed  displacements,  emplacements,  and 
firings  to  enable  it  to  provide  the  desired  increases  in 
responsiveness,  effectiveness,  and  survivability.  Because  these 
requirements  place  a  much- increased  tactical  decision-making 
burden  on  the  HIP  chief-of-section  (COS)  than  had  heretofore 
existed,  the  question  of  whether-or-not  those  at  the  lower  end  of 
the  range  of  13B  capabilities  could  perform  satisfactorily  became 
a  significant  MANPRINT  issue  to  be  addressed  during  the 
development  of  the  HIP.  At  the  time  of  the  request  for  research 
support,  the  data  from  the  HIP  lOTE  had  not  yet  been  analyzed; 
hence,  there  existed  no  directly  applicable  HIP-  or  AFCS-related 
empirical  data  with  which  to  address  the  issue. 
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The  present  abbreviated  assessment  was  conducted  in  response 
to  the  HIP  PM's  request.  A  full-scale  evaluation  was  not 
possible  due  to  the  limited  time  available  to  prepare  a  research 
effort  of  larger  scope,  and  the  limited  availability 
(approximately  12  days)  of  a  functional  HIP  for  the  purpose  of 
conducting  such  research. 

The  severe  time  constraint  and  resource  limitations  which 
existed  precluded  consideration  of  an  extensive  evaluation  of  the 
variety  of  Level  III  or  IV  scenarios  which  could  be  Invoked 
during  field  exercises.  Additionally,  the  time  constraint 
preempted  consideration  of  a  comparative  assessment  evaluation 
approach.  It  was  impossible  to  consider  the  use  of  any  approach 
which  would  have  compared  final  performance  levels  while  using  a 
HIP  in  the  field  because  of  the  extensive  amount  of  time  (and 
instrumentation-related  resources)  required  merely  to  conduct  the 
"test"  portion  of  such  a  process  (i.e.,  in  addition  to  the  time 
required  for  training,  per  se) .  The  consideration  of  an  approach 
which  would  compare  the  performance  of  those  undergoing  ET-based 
training  with  those  who  were  given  some  other  form  of  training 
was  similarly  preempted  due  to  time  and  resource  constraints.  To 
adopt  such  a  strategy  would  have  increased  markedly  both  the 
number  of  soldier-participants  and  the  amount  of  time  required  to 
execute  the  research.  The  latter  (increased  execution  time) 
reflects  the  fact  that  (a)  the  same,  limited  number  of  HIP- 
quallfled  SHE- instructors  would  have  had  to  have  conducted  the 
training  for  both  groups,  and  (b)  the  only  other  previously 
employed  "standard"  HIP  AFCS  training  procedure  was  the  lengthy 
procedure,  entailing  individual  hands-on  instruction,  developed 
for  use  during  the  train-up  period  preceding  the  HIP  lOTE. 

Given  the  circumstances  described,  the  decision  was  made  to 
adopt  an  assessment  approach  which  would  provide  empirical  data 
pertaining  to  the  MANPRINT  domain  in  sufficient  quantity  to 
provide  some  reasonable  expectation  that  inferential  statistics 
could  be  applied  and  yield  statistically  reliable  statements 
regarding  the  outcomes.  Accordingly,  the  research  approach,  as 
described  below,  focused  on  13B  soldiers  with  a  wide  range  of 
aptitudes  during  their  exposure  to  both  Level  I  and  II  tutorials, 
and  during  their  execution  of  a  single  "shoot  heavy"  Level  III 
scenario.  All  training  was  videotaped  and  conducted  on  a  non- 
moving  HIP  with  a  fully  operational  turret. 


Method 


General 

As  discussed  further  in  the  succeeding  section,  the  research 
was  confined  to  an  assessment  of  Levels  I,  II,  and  III  of  the  ET. 
Hands-on  experience  and  performance  assessments  with  the  ET  were 
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limited  to  an  immobile  HIP  with  a  functional  turret  and  gun  tube. 
The  research  assessed  (a)  subjective  evaluations  of  the  perceived 
merits  of  ET  rendered  by  subject  matter  experts  (SMEs)  based  on 
hands-on  experience  with  both  Levels  I  and  II  of  the  Tutorial, 
and  a  single  scenario  from  Level  III,  (b)  subjective  reactions  to 
the  Level  III  scenario  by  individual  HIP-naive  13B  soldiers 
considered  to  be  viable  candidate  Chiefs-of-Section  (COSs)  in 
terms  of  rank  and  experience,  (c)  measures  of  the  hands-on 
performance  (time-for-task-completions  and  accuracy)  of  these 
same  HIP-naive  13Bs  during  their  initial  exposure  to  ET  Levels  I, 
II,  and  III,  (d)  a  detailed  document-based,  screen-by-screen 
review  of  the  Tutorial  portion  of  the  ET  (i.e..  Levels  I  and  II) 
by  ARI  personnel,  and  (e)  observations  of  other  preparatory  and 
procedural  issues  noted  by  ARI  personnel  during  the  preparation 
for,  and  the  conduct  of  the  assessment. 

Assessment  Environment 

The  hands-on  assessments  were  conducted  on  an  actual  HIP 
howitzer  in  a  vehicle  bay  within  the  Gunnery  Department  of  the 
U.S.  Army  Field  Artillery  School.  This  environment  was  chosen 
because  it  was  compatible  with  the  need  to  videotape  the 
performance  of  all  hands-on  performance  by  the  participants.  The 
front  one-half  of  the  howitzer  protruded  through  an  open  door  of 
the  bay  to  the  outside  of  the  building.  During  the  hands-on 
portions  of  the  evaluation,  the  howitzer  itself  did  not  move, 
albeit  the  turret  would  turn  and  the  gun-tube  would  change  in 
elevation  in  response  to  commands  from  the  AFCS  computer,  just  as 
it  would  during  normal  operations.  The  extent  of  such  movement 
was  constrained,  however,  by  prior  calculations  to  assure  that 
the  relative  location  and  size  of  the  target  box  would  preclude 
the  slewing  of  the  gun  tube  into  the  walls  of  the  bay  when 
specific  targets  were  identified  for  engagement  during  the 
execution  of  the  Level  III  scenario. 

SME -Assessment 

Subjects.  Four  subject  matter  experts  (SME's)  were  employed 
to  assess  the  perceived  merits  of  the  ET.  Two  of  the  SMEs  were 
from  the  Gunnery  Department  (GD)  and  two  were  HIP  COSs  assigned 
to  A-2/17  FA.  The  two  GD  SMEs  previously  had  served  as  trainers 
for  the  FA  units  using  the  HIP  during  the  lOTE.  The  COS  SMEs 
were  trained  as  HIP  COSs  and  had  performed  that  role  during  the 
HIP  lOTE. 

On  the  average,  the  gunnery  department  SME's  had  served  as 
13Bs  for  14.3  years,  had  11.3  years  experience  with  field 
artillery  weapon  systems,  and  14  months  experience  with  the  HIP. 
One  gunnery  SME  was  a  staff  Sergeant  (SSG) ,  while  the  other  was  a 
Sergeant  First  Class  (SFC) .  Their  average  age  was  41.5  years. 
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In  contrast,  the  COS  SMEs  had  served  as  13Bs  for  an  average 
of  6.3  years,  had  4.1  years  experience  with  field  artillery 
weapon  systems,  and  4  months  experience  with  the  HIP.  They  were 
both  Sergeants  (SGTs)  and  had  an  average  age  of  27.5  years. 

Procedures .  The  SMEs  received  an  introductory  briefing  on 
the  HIP  ET  from  a  contractor's  representative  who  had  been 
involved  in  the  writing  of  the  software  for  Levels  III  and  IV  of 
the  ET.  This  briefing  outlined  the  manner  in  which  the  system 
was  designed  and  indicated  the  nature  of  the  preparations  and 
calculations  required  before  an  individual  could  begin  training 
in  the  constrained  environment  under  which  the  evaluation  was 
conducted . 

Subsequent  to  receiving  the  introductory  briefing,  the  SMEs 
completed  a  biographical  data  sheet  (see  Appendix  A) .  The  SMEs 
then  collectively  performed  Levels  I  and  II  in  their  entirety, 
and  executed  a  single  "shoot  heavy”  scenario  from  Level  III. 

After  performing  each  Level,  the  SMEs  completed  a  data  collection 
form  containing  18  items  (see  Appendix  B) .  Each  item  consisted 
of  adjective-pairs  separated  by  a  7-point  scale.  The  adjectives 
appeared  at  the  end-points  of  the  scales  as  labels.  The  items 
were  presented  in  random  order.  To  minimize  possible  effects 
associated  with  a  response  bias,  labels  considered  more  positive 
appeared  with  equal  frequency  on  the  left  and  right  sides  of  the 
data  collection  form.  In  general,  the  18  items  addressed  topics 
pertaining  to  the  degree  of  logic,  depth,  and  integration 
evidenced;  the  perceived  worth,  need,  relevance,  and  practicality 
of  the  ET;  the  perceived  quantity  of  the  material  presented,  and 
the  personal  reaction  evoked  (e.g.,  the  degree  of  stress  and 
boredom  inherent  in  the  training  process) . 

COS-Candidate  Assessments 

Subjects.  The  novice  participants  used  in  this  study  were 
eighteen  13B  COS  candidates.  Their  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB)  General  Technical  (GT)  scores  ranged 
from  79  to  121.  Of  the  18  subjects,  10  had  a  GT  score  of  110  or 
above  and  were  assigned  to  the  Categories  I-II  group,  while  the 
remaining  8  had  a  GT  score  below  110  and  were  assigned  to  the 
Categories  III-IV  group. 

The  subjects  assigned  to  Categories  I-II  group  had  served  as 
a  13B  for  an  average  of  4.9  years  and  had  an  average  of  4.2  years 
experience  with  field  artillery  weapon  systems.  There  were  eight 
Corporals  (CPLs)  and  two  SGTs,  with  an  average  age  of  25.6  years. 

The  subjects  assigned  to  Categories  III-IV  group  had  served 
as  a  13B  for  an  average  of  7.0  years  and  had  an  average  of  6.0 
years  experience  with  field  artillery  weapon  systems.  There  were 
five  SGTs  and  three  SSGs,  with  an  average  age  of  28.3  years. 
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Procedures.  On  reporting  to  the  vehicle  bay,  each  subject 
completed  a  biographical  data  sheet  (see  Appendix  A)  and  was 
shown  the  opinion  survey  (see  Appendix  C)  to  be  completed  after 
training. 

The  svibject,  instructor,  and  observer  then  entered  the  HIP 
and  subjects  were  given  a  brief  orientation.  They  were  informed 
that  this  was  to  be  a  test  of  the  adequacy  of  the  ET,  rather  than 
a  test  of  their  ability,  and  that  the  test  would  take 
approximately  two  hours.  Subjects  were  asked  to  try  to  complete 
the  training  without  assistance,  but  were  told  that  if  at  any 
point  they  were  unable  to  proceed  the  instructor  would  be 
available.  Subjects  then  completed  training  Levels  I  and  II. 

The  observer  recorded  errors  made  and  the  instructor  provided 
assistance  as  necessary. 

Prior  to  initiating  Level  3,  subjects  were  given  the  data 
values  to  be  entered  and  a  brief  introduction  to  the  use  of  the 
hydraulics  system.  They  were  also  instructed  on  how  to  use  the 
"load"  and  "lay"  keys  and  warned  not  to  deactivate  the  "training 
on/off"  or  "gun  servo"  switches.  (Note:  This  report  cites 
requirements  for  additional  instruction  as  areas  to  be  addressed 
and  corrected  during  further  developmental  efforts  on  the  HIP 
ET.) 


After  completing  the  data  entry  phase  of  Level  III,  all 
subjects  required  additional  instruction  on  how  to  use  the 
navigational  screens  and  process  the  fire  missions.  The  ET  Level 
III  performance  errors  captured  by  the  system  were  "offloaded" 
through  the  serial  port  to  a  diskette  in  a  portable  computer 
using  CROSSTALK  communication  software.  A  special  cable 
(borrowed  from  the  contractor)  to  interface  between  the  ET  port 
and  the  computer  was  necessary  to  accomplish  the  offloading  task. 

Following  completion  of  the  Level  III  scenario,  subjects 
responded  to  a  survey  designed  to  assess  their  opinion  of  that 
portion  of  the  ET.  The  survey  was  composed  of  11  bipolar 
adjective  pairs  separated  by  a  7-point  scale.  Physical,  mental, 
and  training  aspects  were  evaluated.  Sample  adjective  pairs 
include  relaxing-stressful,  boring-interesting,  and  ineffective- 
effective.  Item  pairs  were  counterbalanced  as  to  their 
desirability. 

The  total  number  of  errors  committed  and  the  times  required 
to  complete  Levels  I  and  II  of  the  tutorial  were  recorded.  On 
Level  III,  the  times  required  to  initialize  the  system  and  enter 
the  ammo  data  were  recorded.  The  offloaded  performance  summary 
data  were  employed  to  assess  errors  made  during  the  Level  III 
scenario. 

6T  scores  were  obtained  after  the  testing  was  completed. 
Analyses  were  conducted  by  mental  category  level,  with  subjects 
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scoring  110  or  above  assigned  to  the  Categories  I-II  group,  and 
subjects  scoring  below  110  assigned  to  the  Categories  IIl-IV 
group. 


Tutorial  Screen  Review  (ET  Levels  I  and  II) 


To  accomplish  a  screen-by-screen  review  of  the  Level  I  and 
II  tutorials,  the  most  recent  existing  documentation  was  sought 
from  the  software  developer.  The  documentation  provided  to  the 
Fort  Sill  Field  Unit  from  the  developer  was  dated  18  January 
1988.  The  hardcopy  representation  of  each  screen,  plus  the 
branching  instructions  provided  for  each  response  option  cited 
for  each  screen  were  individually  examined.  The  review  addressed 
issues  pertaining  to  the  logic  of  the  branching  instructions,  the 
content  of  each  screen,  the  consistency  of  terminology  employed, 
the  manner  of  presentation,  and  other  issues  which  might  affect 
interpretation  by  the  COS  candidate  trainees. 


Results 


SME -Assessment 

In  general,  the  reactions  indicate  that  the  ET  is  viewed 
positively;  i.e.,  the  perceptions  were  that  ET  was  logical, 
practical,  needed,  and  relevant.  There  were  no  statistically 
significant  differences  among  the  three  ET  Levels  addressed.  The 
overall  mean  SMEs*  perceptions  are  summarized  in  Table  1.  (Note: 
The  manner  in  which  these  findings  are  presented  does  not  reflect 
the  construction  of  the  data  collection  form  [see  Appendix  B] , 
rather  they  are  grouped  to  reflect  similarities  in  content  and 
placement  of  adjectives.  Labels  having  more  "favorable"  [rather 
than  "less  favorable"]  or  "greater"  [rather  than  "lesser"] 
connotations  have  been  re-arranged  to  all  appear  on  the  right 
side  of  the  table.) 

COS-Candidate  Assessment 

Subjective  perceptions.  The  opinion  survey,  administered 
only  after  Level  III  and  pertaining  to  that  level,  was  analyzed 
by  comparing  the  participants'  mean  responses  with  the  neutral 
point  for  each  item  pair.  The  value  4  was  taken  as  the  neutral 
point  for  each  7-point  scale.  The  ratings  revealed  to  be 
significantly  different  from  the  neutral  point  were  those  which 
found  the  ET  to  be  interesting  (t[17]  =  9.78,  e<.01),  effective 
(j;[17]  ■  3.83,  E<*01)/  satisfactory  (t[17]  «  4.44,  E<«01)» 
requiring  a  high  degree  of  coordination,  (J^[17]  «  3.13,  e<*01), 
and  paced  too  fast,  (t[l7]  =  2.12,  e<*05).  None  of  the  other 
ratings  differed  significantly  from  the  scale  midpoint.  The 
perceptions  did  not  differ  significantly  (e>*05)  between 
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categories  I-II  and  III-IV.  Table  2  depicts  the  mean  response 
magnitudes  for  each  item. 

Objective  performance  measures.  The  Level  I  tutorial  is  an 
elementary  one.  It  can  be  successfully  completed  in  a  total  of 
15  screens  (of  which  three  are  used  merely  to  provide  the  word 
"CORRECT"  as  feedback,  and  one  is  used  to  indicate  that  Level  I 
has  been  completed) .  Level  II  is  a  much  more  comprehensive 
tutorial,  involving  more  than  eight  times  as  many  screens  to  be 
completed.  Accordingly,  to  obtain  a  more  reliable  estimate  of 
error  performance,  the  number  of  errors  committed  in  the  Level  I 
tutorial  was  added  to  the  number  of  errors  in  Level  II  tutorial. 

For  all  of  the  measures  employed,  the  mean  performance  for 
those  with  GT  scores  110  or  higher  was  better  (i.  e.,  they  made 
fewer  errors  and  completed  each  task  more  quickly)  than  the  mean 
performance  for  individuals  with  lower  GT  scores.  The 
differences  were  statistically  significant  for  initialization 
time,  t(17)  =  2.68,  p<.02,  and  ammo  entry  time,  t(17)  =  2.63, 
p<.02.  The  mean  times  to  complete  each  task  and  the  mean  number 
of  total  errors  committed  within  the  Level  I  and  II  Tutorials 
(combined)  are  shown  in  Table  3  for  the  two  category  groupings. 
The  findings  are  also  depicted  in  Figures  1-5. 
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Table  1 


SME-'Assessnen't  of  bhe  Embedded  Trainer 


ILLOGICAL 

* — 

— * — 

— 

•*- 

-x- 

-* - 

— * — 

LOGICAL 

SUPERFICIAL 

* — 

— * — 

— 

•X— 

•*- 

— 

-* - 

— 

— * 

IN-DEPTH 

DISORGANIZED 

* — 

— *- 

- - 

•*. 

— 

-*-x- 

— * — 

— * 

ORGANIZED 

POORLY 

WELL 

THOUGHT  OUT 

* — 

— 

—  —  - 

•*- 

-X- 

— 

— * — 

— * 

THOUGHT  OUT 

ARTIFICIAL 

* — 

— * — 

— *- 

— 

'X- 

— 

— * — 

— * 

REALISTIC 

UNNECESSARY 

* — 

— *- 

— 

— 

-* — 

-x*— 

— * 

NECESSARY 

INEFFECTIVE 

* — 

— * — 

— *- 

— 

— 

-*x- 

— * — 

— * 

EFFECTIVE 

IMPRACTICAL 

* — 

— *— 

— *• 

— 

— 

X*— 

— * — 

— * 

PRACTICAL 

IRRELEVANT 

* — 

— *. 

— 

-*• 

— 

-*x- 

— * — 

— * 

RELEVANT 

UNSATISFACTORY 

— *. 

— 

— X 

— 

— * 

SATISFACTORY 

RELAXING 

X- 

— * 

STRESSFUL 

PACE  TOO  SLOW 

— *. 

— X 

.*■ 

— 

— >* 

PACE  TOO  FAST 

BORING 

«  — 

— *■ 

— 

-X- 

-* — 

INTERESTING 

UNFAMILIAR 

* — 

— *• 

— 

-*■ 

— 

-*x- 

— * 

FAMILIAR 

TOO  EASY 

* — 

— *. 

-X- 

-*■ 

— 

.* — 

— * — 

TOO  DIFFICULT 

TOO  LITTLE 

* — 

— *. 

-X- 

-*■ 

— 

.* — 

— * 

TOO  MUCH 

TOO  SIMPLE 

* — 

— *• 

— X 

-*■ 

— 

— 

— * 

TOO  COMPLEX 

TOO  SHORT 

* — 

— * — 

— * — 

— *• 

-X- 

-*■ 

— 

— 

— * — 

— * 

TOO  LONG 
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Table  2 


COS-Candidate  Assessment  of  the 

Embedded 

Trainer 

Subscales 

M 

SD 

Physical  Aspects 

Not  Fatiguing  -  Fatiguing 

3.39 

1.79 

Relaxing  -  Stressful 

4.28 

1.49 

Low — High  Coordination 
Recjuirement 

5.17** 

1.58 

Mental  Aspects 

Too  Easy  -  Too  Difficult 

3.78 

1.06 

Boring  -  Interesting 

6.39** 

1.04 

Familiar  -  Unfamiliar 

3.33 

1.50 

Relaxing  -  Stressful 

3.95 

1.66 

Training  Aspects 

Pace  too  -  Pace  too 

Slow  Fast 

4.39* 

.78 

Ineffective  -  Effective 

5.61** 

1.79 

Too  Simple  -  Too  Complex 

3.94 

.87 

Unsatisfactory  -  Satisfactory 

5.78** 

1.70 

Notes;  A  7-polnt  scale  with  labeled  endpoints  was 
used.  The  labels  for  each  scale  are  cited.  The  left 
label  corresponds  to  the  scale  value  1. 

H  *  18. 

Asterisks  denote  levels  of  statistical  significance: 

*  denotes  p  <  .05;  **  denotes  p  <  .01. 


I 


10 


Table  3 


Means  and  standard  Deviations 
by  Category 

for  Training 

Performance 

Training  Performance 

Measures 

Cateaories 

I-II* 

III-IV® 

Level  1  Completion  Time 

n 

6.29 

8.26 

sn 

2.43 

2.96 

Level  2  Completion  Time 

M 

32.40 

35.27 

SjB 

9.48 

10.21 

Errors  Committed  on  Levels  1 

&  2 

3.90 

7.00 

3.03 

8.52 

Initialization  Time 

M 

13.13 

20.78* 

sn 

4.99 

7.14 

Ammo  Entry  Time 

15.02 

19.69* 

SC 

4.19 

3.11 

Notes ;  *11=  10.  ‘*11=  8. 

*  Level  of  statistical  significance:  £  <  .02. 
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FIGURE  2.  TIm  raquirad  to  complete  Level  II  trelnlng  by  mental  category  grouping 
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ERRORS 


1 


ERROR  SCORES 


LEVELS  I  AND  II 

CATEGORIES  III-IV 


Figure  3.  Errors  cosmitted  on  Levels  I  and  II  by  nental  category  grouping. 
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Figure  4.  Syetea  Initialization  tiae  requirement  by  mental  category  grouping. 
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Tutorial  Screen  Review  (Levels  I  and  II) 

The  following  conunents  characterize  the  results  of  the  review  of 
the  screens  used  in  the  Level  I  and  II  tutorials.  (The  detailed 
comments  appear  in  Appendix  D.) 

Computer  literacy  issues.  Given  the  degree  of  computer 
literacy  that  may  exist  (i.e.,  next  to  zero  in  many  instances),  one 
recommendation  is  that  the  soldier  be  provided  with  an  initial 
functional  definition  of  pertinent  terms  which  appear  (or  are 
recommended  in  this  report  to  appear)  within  the  ET  screens.  There 
is  a  need  for  a  description  of  the  functional  differences  between, 
and  the  locations  of  the  "hardkeys"  and  "SOFTKEYS,"  at  the  outset 
of  the  tutorial.  There  is  also  a  need  to  introduce  and  define  the 
terms  "MENU"  (to  refer  to  the  lists  of  items  from  which  selections 
are  to  be  made) ,  "HIGHLIGHT"  (to  refer  to  the  reverse  video  used  to 
indicate  the  position  of  the  "cursor"  when  the  arrow  keys  are  used 
to  designate  an  option  or  item  selected  from  a  menu) ,  and  a 
specialized  meaning  of  the  word  "TYPE"  (to  refer  to  key  presses  on 
the  numeric  keypad)  early  in  the  sequence  of  screens.  The  term 
"SCREEN"  is  preferred  to  the  term  "DISPLAY"  throughout  (the  term 
DISPLAY  within  the  human  factors  area  has  a  more  generic  meaning; 
i.e.,  there  are  "auditory  displays,"  "tactile  displays,"  as  well  as 
"visual  displays"). 

Multiple  commands.  Potential  confusion  exists  on  many  screens 
in  Levels  I  and  II  because  two  commands  appear  within  a  single 
screen.  Evidence  of  such  confusion  appeared  during  the  actual 
hands-on  assessments.  As  cited  below,  such  screens  typically 
include  both  (a)  a  description  of  an  action  which  is  to  be  executed 
when  the  succeeding  screen  is  presented  (e.g.,  SELECT  BAUD  RATE), 
and  (b)  a  description  of  the  action  necessary  to  terminate  the 
current  screen  (e.g.,  with  the  words  "PRESS  CONTINUE  TO  PROCEED"). 

Initial  kev  definitions.  The  nature  ("soft"  or  "hard," 
particularly  for  the  "soft"  keys)  and  location  of  a  key  to  be 
pressed  is  not  provided  when  the  key  is  first  used.  There  is  a 
need  to  provide  such  information  at  the  initial  encounter. 


Consistency  of  terminology.  There  were  instances  wherein  the 
identifying  names  and  titles  used  within  screens  were  inconsistent 
with  the  names  and  titles  actually  appearing  on  the  hardware,  the 
screens,  and  the  menu-related  tasks  and  subtasks  to  which  they 
refer.  Within  the  main  body  of  the  instructional  screens,  there 
were  names  and  titles  which  reflect  errors  of  two  principal  types. 
One  type  did  not  reflect  the  use  of  the  full  name  or  title  that 
actually  appeared  (this  occurred  most  frequently  when  referring  to 
titles  of  succeeding  screens) .  The  second  type  as  the  reverse  of 
the  first;  i.e.,  the  screen  cited  the  full  name  for  a  referent 
which  actually  appeared  in  an  abbreviated  form  on  another  screen 
(especially  when  referring  to  items  of  screen  menus  and  keys) . 
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Differentiation  of  referents.  The  referents  employed  in  the 
body  of  many  screens  cited  the  titles  of  succeeding  screens,  items 
on  menus  appearing  on  succeeding  screens,  and  keys  to  be  pressed. 

To  make  it  easier  for  operators  to  identify  and  discriminate  the 
referent,  there  is  a  need  to  employ  specific  and  consistent  ways  of 
identifying  and  differentially  indicating  to  the  trainer  and 
trainee  that  the  referent  is  a  SCREEN  TITLE,  a  (soft  or  hard)  KEY 
to  be  pressed,  or  an  ITEM  to  be  selected  from  a  menu  presented  on  a 
screen.  In  the  absence  of  specific  knowledge  regarding  the 
availability  of  software  capabilities  which  might  otherwise  be 
employed  to  accomplish  these  goals,  it  may  possible  to  use  bold 
characters  (or  reverse  video)  in  conjunction  with  double  quotation 
marks  to  refer  to  a  "SCREEN  TITLE,"  bold  characters  (or  reverse 
video)  within  single  quotation  marks  to  refer  to  a  'MENU  ITEM, '  and 
bold  letters  (or  reverse  video)  alone  to  identify  a  KEY  to  be 
pressed. 

Deletion  of  "DIRECTIONS"  subtitle.  The  inclusion  of  a 
separate  line  stating  "DIRECTIONS"  (on  the  initial  line  within  the 
main  body  of  the  screen)  adds  very  little  to  the  tutorial  and 
requires  memory  space  which  might  be  otherwise  used.  The  term 
"DIRECTIONS"  could  be  eliminated  from  the  top  of  screens  without 
adversely  affecting  operator  performance. 

Deletion  of  separate  "CORRECT"  screens.  Whenever  a  selection 
is  to  be  made  by  a  trainee,  and  the  trainee  makes  a  correct 
response,  a  separate  screen  displays  the  word  "CORRECT"  along  with 
an  instruction  to  "PRESS  CONTINUE  TO  PROCEED."  In  contrast  to  this 
procedure,  whenever  an  incorrect  response  is  made,  the  initial  line 
of  the  succeeding  screen  identifies  the  response  as  having  been 
incorrect  (e.g.,  YOU  HAVE  MADE  AN  INCORRECT  SELECTION)  and  devotes 
the  remainder  of  the  screen  to  providing  additional  information  to 
assist  the  trainee  to  determine  the  correct  response.  The  latter 
procedure,  the  one  pertaining  to  feedback  for  incorrect  responses, 
is  a  more  efficient  approach  in  that  better  use  is  made  of  the 
available  memory.  The  use  of  a  separate  screen  to  display  the  word 
"CORRECT"  appears  unwarranted.  If  "CORRECT"  were  placed  at  the 
beginning  of  the  succeeding  screen  along  with  a  transitional  word 
or  phrase  to  introduce  the  next  topic  (for  example:  CORRECT!  NOW 
PRESS  THE  CLR  KEY  TO  ....),  then  the  remainder  of  the  screen  could 
be  used  to  provide  other,  more  useful  information  (see  Appendix  D) . 

Findings 

AFCS  interface  equipment,  software,  and  instructions.  The 
statement  of  the  Required  Operational  Capability  (ROC)  and  the 
dociunentatlon  provided  are  consistent  in  stating  the  need  for  a 
means  of  "offloading"  performance  data  from  the  AFCS.  However,  the 
equipment  (to  include  relevant  interface  items  needed) ,  the 
software,  and  the  instructions  required  to  accomplish  this 
operation  were  not  available.  The  necessary  equipment  was  not 
cited  in  the  TOE,  nor  were  relevant  procedures  found  in  any  draft 
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operator's  manual  or  handbook.  (The  actual  equipment  and 
instructions  employed  to  conduct  the  assessment  were  provided  by 
representatives  from  the  PM's  office  and  the  developer.) 

Preliminary  data  requirements,  calculations,  and  information 
needs .  The  circumstances  under  which  the  present  assessment  was 
undertaken  were  described  earlier  in  the  METHOD  section.  While  ti  -i 
assessment  environment  does  present  some  unusual  constraints 
(versus  undertaking  same  in  a  field  training  area) ,  the 
preparations  necessary  did  surface  the  fact  that  information 
regarding  the  existing  location  of  the  howitzer  and  the  anticipated 
location  of  the  target  area  must  be  known  and  relevant  calculations 
performed  before  training  can  safely  begin. 

During  the  present  effort,  the  SMEs  who  monitored  the  COS 
candidate's  efforts  and  answered  questions  posed  by  same  had  to 
accomplish  a  number  of  otherwise  unidentified  tasks  before  the 
trainee  could  begin  the  ET  process.  These  tasks,  as  indicated  by 
the  PM  and  the  developer's  representatives,  consisted  of  the 
following:  power-up  the  howitzer  and  select  the  ET  training  option, 
select  Level  III,  identify  the  scenario  to  be  run,  select  the 
"Performance  Save"  method,  enter  the  current  Grid  Zone  to 
initialize  the  Dynamic  Reference  Unit,  and  enter  three  maps  (the 
training  area  map,  the  howitzer  area  map,  and  the  target  area  map) . 
Site  data  and  mask  information  were  also  required. 

In  addition  to  these  tasks,  the  trainee's  supervisor  may  have 
to  perform  the  calculations  to  assure  that  the  relative  sizes  and 
geometries  of  the  howitzer  and  target  area  maps  will  permit  safe 
operation  of  the  howitzer  and  are  consistent  with  the  additional 
information  (locations)  to  be  employed  during  the  conduct  of  Level 
III  or  IV  scenarios. 

Transition  from  Level  II  to  Level  III.  The  present  Level  I 
and  II  Tutorial  materials  do  not  provide  sufficiently 
general izable  information,  examples,  and  practice  opportunities  to 
enable  the  COS  candidate  trainee  to  successfully  transition  to  a 
Level  III  scenario  without  a  significant  amount  of  support  from  an 
on-site  COS-qualified  individual.  Considerable  instructor  support 
was  required  at  the  beginning  of  Level  III;  e.g.,  instructions 
regarding  the  hydraulics  system  activation,  and  positioning  of  the 
ODH  SERVO  SWITCH  and  information  regarding  the  GUM  SERVO  INDICATOR 
were  required  for  all  participants.  Additionally,  confusion  was 
evidenced  by  all  COS  candidate  trainees  at  their  initial  encounters 
with  the  navigational  screens  ("STEER  TO"  and  "EMPLACE")  and  the 
initial  fire  mission  screen.  These  are  not  addressed  previously 
during  the  Level  I  and  II  Tutorial. 

The  COS  trainees'  supervisors  must  also  address  the  following 
prior  to  beginning  a  Level  III  or  IV  scenario:  define  restart 
locations  to  be  used,  provide  a  list  of  ammunition  to  be  used, 
identify  relevant  radio  communication  parameters  to  be  employed. 
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Discussion 


The  subjective  reactions  of  both  the  SMEs  and  those 
participating  as  COS  candidate  trainees  reflected  quite  positively 
on  the  ET.  With  the  exception  of  a  perception  that  the  pace  of  the 
Level  III  instiruction  was  ”too  fast,”  there  was  no  instance  of  any 
perception  that  was  critical  of  the  ET  concept  or  the  system. 

One  of  the  major  concerns  appearing  in  the  HIP  MANPRINT  area 
is  that  pertaining  to  the  ability  of  the  entire  target  population 
to  be  able  to  adequately  perform  the  COS  functions.  Clearly  the 
present  sample  size  is  small,  and,  as  a  consequence,  the  results 
cannot  be  considered  definitive.  However,  the  findings  and 
observations  permit  some  degree  of  plausible  inference  to  be  made. 
While  it  is  emphasized  that  virtually  all  of  the  participants 
appeared  to  be  very  sincerely  trying  to  perform  the  task,  it  was 
clear  from  the  perseveration  evidenced  by  one  of  the  participants 
that  it  would  require  extensive  and  intensive  training  to  attain 
acceptable  levels  of  performance.  For  two  of  the  others,  the  task 
appeared  to  represent  a  highly  demanding  undertaking,  and 
considerable  assistance  was  required  to  complete  it. 

Among  the  objective  time  and  accuracy  performance  data,  the 
present  research  also  yielded  consistent  findings  of  superior 
performance  by  those  in  the  Category  I-II  group  relative  to  that 
evidenced  by  those  in  the  Category  III-IV  group.  Hence,  the 
Informal  observations  cited,  when  coupled  with  the  reliable 
empirical  performance  differences  encountered,  suggests  that  there 
is  reason  for  continuing  concern  regarding  whether  or  not  the 
entire  target  population  of  13B  COS  candidates  will  be  able  to 
function  at  acceptable  levels  of  performance  in  the  role  of  HIP 
COS.  The  need  for  further  research  is  indicated. 

As  regards  the  overall  level  of  implementation,  it  was  the 
consensus  of  the  SMEs  and  the  present  authors  that  the  absence  of 
adequate  information  and  instruction  at  the  beginning  of  the 
Tutorial  and  at  the  transition  between  Level  II  of  the  Tutorial  and 
the  Level  III  scenario  constitutes  a  significant  drawback  to  the  ET 
in  its  present  state  of  development.  The  present  Level  I  and  II 
Tutorials  do  not  address  the  preparatory  requirements  at  all. 
Neither  do  they  provide  sufficiently  generalizable,  operationally- 
relevant  or  "operationally-similar”  information,  examples,  and 
practice  opportunities  to  enable  the  COS  candidate  trainee  to 
successfully  transition  to  a  Level  III  scenario  without  a 
considerable  amount  of  support  from  an  on~site  COS-qualif led 
individual . 

At  the  present  state  of  development,  the  HIP  ET  requires  the 
presence  of  a  HIP-qualified  instructor  to  assist  the  student  during 
the  conduct  of  training.  The  HIP  ET  could  achieve  a  greater  degree 
of  self-teaching  capability  if  additional  information  and 
instructions  were  provided  regarding  the  "pre-Level  I”  preparations 
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required  before  beginning  a  training  session;  incorporating  more 
readily  generalizable  information,  examples,  and  practice 
opportunities  within  Level  II;  and  adding  additional  transitional 
information  between  Levels  II  and  III.  Such  would  have  to  be 
either  incorporated  in  the  software  itself  (perhaps  prohibitive  in 
lieu  of  existing  memory  size  limitations)  or  in  separate-but- 
necessary,  closely  integrated  operator's  and  trainer's  handbooks  to 
accompany  the  ET. 

While  the  Level  IV  simulation  capabilities  were  not  addressed 
in  the  present  evaluation  due  to  the  existing  resource  and  time 
constraints,  the  contractor's  description  of  same  clearly  suggest 
that  the  HIP  ET  will  provide  a  markedly  enhanced  unit  training 
capability — stretching  the  units'  training  capabilities  beyond 
circumstances  that  they  would  otherwise  be  able  to  address. 
Moreover,  the  ET  will  provide  field  units  with  a  crucial  element 
that  is  largely  absent  in  the  sustainment  training  process:  the 
capability  to  obtain  objective  data  for  the  purpose  of  providing 
performance-related  feedback  to  their  soldiers.  Hence,  while  it  is 
the  case  that  the  present  research  has  encountered  areas  wherein 
levels  I,  II,  and  III  of  the  HIP  ET  are  presently  in  need  of 
revision  and  augmentation,  the  HIP  ET  is  viewed  as  representing  a 
much-desired  and  powerful  concept  worthy  of  the  additional  effort 
required  to  bring  it  to  fruition. 

Conclusions 

Predicated  upon  the  findings  and  observations  of  the  present 
research,  the  following  conclusions  apply: 

1.  The  subjective  perceptions  provided  indicate  that  the  HIP 
ET  was  well  received  by  both  the  SMEs  and  those  participating  as 
COS  candidates. 

2.  Objective  performance  by  those  having  ASVAB  GT  scores 
placing  them  in  higher  Categories  (I  and  II)  was  consistently 
better  than  that  for  those  having  GT  scores  placing  them  in  lower 
Categories  (III  and  IV) . 

3.  There  is  a  need  for  revisions  to  the  content  of  the 
screens  presented  to  accommodate  the  low  degree  of  computer 
literacy  anticipated  in  the  target  population,  reduce  confusion 
associated  with  the  appearance  of  multiple  commands  within  a  single 
screen,  provide  clearer  identification  of  keys  and  differentiation 
of  referent  terms,  obtain  greater  consistency  in  the  terminology 
employed,  and  achieve  greater  effectiveness  of  screen  usage  through 
deletion  of  recurrent  subtitles  and  separate  "CORRECT"  screens. 

4.  There  is  a  need  to  add  either  additional  screens  to  the 
existing  ET  or  provide  supplemental  manuals  for  the  trainer  and  the 
trainee  to  meet  requirements  not  now  addressed  regarding 
computations  to  be  performed  and  information  required  to  perform 
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all  tasks  adequately;  e.g.,  the  information  needed  prior  to 
beginning  an  ET  session  and  in  transitioning  from  the  Level  II 
Tutorial  to  the  Level  III  scenario,  information  regarding  the  use 
of  the  navigational  screens  and  how  to  process  fire  missions,  and 
instructions  regarding  the  set-up  and  use  the  operator  performance 
downloading  capability. 

5.  There  is  a  need  to  assure  that  the  using  units  possess  the 
software,  documentation,  and  hardware  needed  to  download 
performance  summaries. 


Recommenda t ions 

It  is  recommended  that: 

1.  Research  be  performed  to  further  investigate  the 
relationship  between  mental  categories  and  level-of-performance  and 
to  assess  the  adequacy  of  COS  performance  at  the  lower  ranges  of 
capabilities  anticipated  for  the  13B  COS  candidate  population. 

2.  The  existing  HIP  ET  software  be  modified  to  enhance  the 
interpretability  of  the  screens  (see  Appendix  D) . 

3.  Additional  screens  and  closely  integrated  HIP  trainer  and 
trainee  user  manuals  be  developed  to  provide  the  additional 
information  and  describe  the  calculations  required  both  to  begin  an 
ET  training  session  to  effectively  transition  between  the  Level  II 
Tutorial  and  Level  III  scenarios,  and  (as  a  minimum)  to  minimize  or 
(preferred)  to  eliminate  the  need  for  other  instructor-provided 
information  or  guidance  as  was  evidenced  during  the  course  of  this 
research. 

4.  Using  unit  be  provided  the  hardware,  software,  and 
docximentation  necessary  to  support  the  downloading  of  performance 
summary  information. 

5.  Additional  research  be  undertaken  to  similarly  assess  the 
Impact  of  the  recommended  revisions  to  the  system. 


20 


I 


References 

Finley,  D.  L. ,  Alderman,  I.  N.,  Peckham,  D.  S.,  and  Strasel,  H.  D. 
(1988) .  Implementing  embedded  training  fET> ;  Volume  1  of  10; 
Overview  (ARI  Research  Product  88-12) .  Alexandria,  VA:  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences.  (AD  A201  401) 

Headquarters,  Department  of  the  Army.  (1987) .  Manpower  and 

Personnel  Integration  (MANPRINT^  in  the  Material  Acouisition 
Process  (AR  602-2) .  Washington,  DC:  Author. 


21 


APPENDIX  A 


BIOGRAPHICAL  DATA  SHEET 
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BIOGRAPHICAL  DATA  SHEET 


Name:, 

Date:, 

Unit: 


SSN: 


Time: 


Primary  MOS: _ 

Rank:  Date  of  Rank: 


Length  of  time  in  unit:_ 
_  Secondary  MOS  (if  any) 
_  Age: _ 


(mo. ) 


Date  of  graduation  from  13B  AIT: 
Total  months  of  service  as  13B: 


Indicate  how  many  months  of  "hands-on"  experience  you  have  had  with 
each  of  the  weapon  systems  below  (if  none,  enter  zero:  0) : 


WEAPON  SYSTEM  MO.  MONTHS 

M109  SELF-PROPELLED  155mm  HOWITZER:  _ 

(excluding  HIP) 

M109  SELF-PROPELLED  "HIP"  HOWITZER:  _ 

M102  TOWED  105mm  HOWITZER:  _ 

MHO  SELF-PROPELLED  8-Inch  HOWITZER:  _ 

OTHER: 
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APPENDIX  B 


SHE  SUBJECTIVE  ASSESSMENT 
DATA  COLLECTION  FORM 


B-1 


Describe  your  reaction  to  LEVEL  _  of  the  EMBEDDED  TRAINER  by 

placing  an  "X"  along  the  line  provided  between  each  of  the  pairs  of 
words  appearing  below: 


LOGICAL  * - * - * - * - * - * - * - *  ILLOGICAL 

TOO  EASY  * - * - * - * - * - * - * - *  TOO  DIFFICULT 

REALISTIC  * - * - * - * - * - * - * - *  ARTIFICIAL 

STRESSFUL  * - * - * - * - * - * - * - *  RELAXING 

SUPERFICIAL  * - * - * - * - * - * - * - *  IN-DEPTH 

PRACTICAL  * - * - * - * - * - * - * - *  IMPRACTICAL 

DISORGANIZED  * - * - * - * - * - * - * - *  ORGANIZED 

PACE  TOO  FAST  * - * - * - * - * - * - * - *  PACE  TOO  SLOW 

SATISFACTORY  * - * - * - * - * - * - * - *  UNSATISFACTORY 

FAMILIAR  * - * - * - * - * - * - * - *  UNFAMILIAR 

TOO  SIMPLE  * - * - * - * - * - * - * - *  TOO  COMPLEX 

NECESSARY  * - * - * - * - * - * - * - *  UNNECESSARY 

INEFFECTIVE  * - * - * - * - * - * - * - *  EFFECTIVE 

TOO  LONG  * - * - * - * - * - * - * - *  TOO  SHORT 

WELL  POORLY 

THOUGHT  OUT  * - * - * - * - * - * - * - *  THOUGHT  OUT 

INTERESTING  * - * - * - * - * - * - * - *  BORING 
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IRRELEVANT  * - * - * * - * * - * - *  RELEVANT 

TOO  MUCH  * - * - * * - * * - * - *  TOO  LITTLE 

Do  you  have  any  other  comments  regarding  LEVEL  Z  Embedded 

Training? _ 


APPENDIX  C 


COS  CANDIDATE  OPINION  SURVEY 
DATA  COLLECTION  FORM 
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Describe  your  reaction  to  the  training  you  have  just  received  on 
LEVEL  3  of  the  EMBEDDED  TRAINER  by  placing  an  "X”  along  the  line 
provided  between  each  of  the  pairs  of  words  appearing  below: 


NOT  FATIGUING 

* — 

— 

PHYSICAL  ASPECTS 

-* — 

— * 

FATIGUING 

STRESSFUL 

* — 

— *. 

- * - 

.* - * - 

— 

— * 

RELAXING 

HIGH 

COORDINATION 

* — 

— 

_ * _ 

.* - * - 

-* — 

— * 

LOW 

COORDINATION 

REQUIREMENT 

TOO  EASY 

* — 

— *. 

MENTAL 

ASPECTS 

.* - * - 

-* — 

— * — 

— * 

REQUIREMENT 

TOO  DIFFICULT 

INTERESTING 

* — 

— *. 

■ - 

.* - * - 

— 

— * 

BORING 

FAMILIAR 

* — 

— 

- - 

.* - * - 

— 

— * 

UNFAMILIAR 

STRESSFUL 

*— 

— *■ 

- * - 

.* - * - 

-* — 

— * 

RELAXING 

PACE  TOO  FAST 

*— 

TRAINING  PROCESS 

— 

— * 

PACE  TOO  SLOW 

INEFFECTIVE 

* — 

— *. 

- 

.* - * - 

— 

— * 

EFFECTIVE 

TOO  SIMPLE 

* — 

— *. 

- - - 

-* — 

— * 

TOO  COMPLEX 

SATISFACTORY 

* — 

— *. 

_ *_ _ 

.* - * - 

— 

— * 

UNSATISFACTORY 

Do  you  have  any  other  cominents  regarding  the  training  you  have 
just  received? _ 


(Continue  as  needed  on  the  other  side.) 
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APPENDIX  D 


RESULTS  OF  FRAME-BY-FRAME  REVIEW  OF 
TUTORIAL  LEVELS  I  AND  II 
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COMMENTS  ON  LEVEL  I  and  II  TUTORIALS 

Note:  The  following  comments  pertain  to  documentation  dated  18 
January  1988.  This  documentation  was  provided  to  the  ARI  Fort  Sill 
Field  Unit  as  being  the  most  recent  version  available  applicable  to 
the  contents  of  the  TUTORIALS. 

Screen  No.  Comment 

GENERAL  1.  Given  the  degree  of  computer  literacy  that  may  exist 
(i.e.,  next  to  zero  in  many  instances),  recommend  that  a 
description  of  the  functional  differences  and  locations  of  the 
"hardkeys"  and  "SOFTKEYS'*  be  provided  at  the  outset  of  the 
Tutorial.  Additionally,  the  introduction  and  definition  of  the 
term  "MENU"  (to  refer  to  the  lists  of  items  from  which  selections 
are  to  be  made,  e.g.,  see  T0118  comment  #3),  "HIGHLIGHT"  (to  refer 
to  the  reverse  video  used  to  indicate  the  position  of  "cursor"  when 
the  arrow  keys  are  used  to  designate  an  option  selected  from  a 
menu,  e.g.,  see  T0161  comment),  and  a  specialized  meaning  of  the 
word  "TYPE"  (to  refer  to  key  presses  on  the  numeric  keypad,  e.g. , 
T0156  comments)  should  be  provided  at  the  beginning  of  the 
Tutorial . 

GENERAL  2.  Potential  confusion  exists  on  several  frames  in  Levels 
I  and  II  because  two  commands  appear  within  a  single  screen.  As 
cited  below,  such  screens  typically  include  both  (a)  a  description 
of  an  action  which  is  to  be  executed  when  the  succeeding  screen  is 
presented  (e.g.,  SELECT  RESTART),  and  (b)  a  description  of  the 
action  necessary  to  terminate  the  current  screen  (e.g.,  with  the 
words  "PRESS  CONTINUE  TO  PROCEED").  Recommend  considering  the 
adoption  of  one  or  both  of  the  following  practices  to  unambiguously 
differentiate  between  the  action  to  be  executed  on  the  succeeding 
screen  from  the  action  necessary  to  terminate  the  presentation  ,of 
the  presently  existing  screen: 

1.  When  addressing  actions  to  be  executed  on  the  following 

screen,  precede  the  description  of  such  deferred  actions  with  the 
words  "ON  THE  NEXT  SCREEN,  SELECT  _ ." 

2.  Precede  and  follow  the  line  of  text  designating  the  actual 
key  to  be  pressed  to  terminate  the  current  screen  with  a  double 
asterisk,  double  dash,  etc.  Within  that  instruction,  consistently 
use  BOLD  (or  reverse  video)  letters  to  identify  the  key  to  be 
pressed  (such  is  inconsistently  done  now,  e.g.,  compare  T1002  and 
T1003) .  Example:  **  PRESS  CONTINUE  TO  PROCEED  ** 

GENERAL  3.  Consider  deleting  the  term  "DIRECTIONS"  from  the  top  of 
most  screens;  it  adds  very  little  to  the  tutorial  and  requires 
memory  space  which  might  be  otherwise  used.  If  this  practice  is 
adopted,  the  deletion  should  be  consistent  throughout  the  ET 
package. 
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GENERAL  4.  Recommend  that  at  the  first  time  a  specific  key  is 
cited  to  be  pressed,  the  nature  ("soft”  or  "hard,”  particularly  for 
the  "soft"  keys)  and  location  of  the  key  should  be  identified. 

GENERAL  5.  Care  should  be  taken  to  assure  that  the  names  used 
within  screens  are  consistent  with  those  actually  appearing  on  the 
hardware,  the  screens,  and  the  tasks  and  subtasks  to  which  they 
refer,  e.g.,  within  TOllO  there  is  a  reference  to  TOlll  as  "NETWORK 
ACCESS;"  the  actual  title  of  TOlll  is  "NET  ACCESS". 


GENERAL  6.  Use  specific  and  consistent  ways  to  differentially 
indicate  to  the  reader  that  you  are  referring  to  a  SCREEN,  a  (soft 
or  hard)  KEY  to  be  pressed,  or  an  item  to  be  selected  from  a  menu 
presented  on  a  screen;  e.g.,  consider  the  use  of  bold  (or  reverse 
video)  characters  in  conjunction  with  double  quotation  marks  to 
refer  to  a  "SCREEN  TITLE,"  bold  (or  reverse  video)  characters 
within  single  quotation  marks  to  refer  to  a  'MENU  ITEM, '  and  bold 
(or  reverse  video)  letters  alone  to  identify  a  KEY  to  be  pressed. 


GENERAL  7.  Consider  deleting  the  practice  of  using  separate 
screens  to  provide  CORRECT  feedback.  Incorporate  CORRECT  at  the 
beginning  of  the  succeeding  screen  and,  if  needed,  include  a 
transitional  word  or  phrase  to  introduce  the  next  topic.  Rationale: 
Conservation  of  memory. 

Examples : 


TlOll  The  initial  line  should  read:  CORRECT. 
MAKING  . . . 

T1015  The  initial  line  should  read:  CORRECT. 
CLR  KEY  to  .... 

T0141  The  initial  line  should  read  CORRECT. 
THE  PROPELLANT  TEMPERATURE  ... 


NOW  PRACTICE 
NOW  PRESS  THE 
NOW  INITIALIZE 


Note:  The  comments  provided  above  are  intended  to  reflect 
characteristics  which  are  present  in  multiple  locations  throughout 
the  dociimentation.  While,  in  the  spirit  of  attempting  to  expedite 
subsequent  revisions,  an  effort  will  be  made  to  identify  each 
applicable  occurrence  (via  an  abbreviated  reference  to  the 
appropriate  general  comment  in  the  following  manner:  Gl,  G2,  ...), 
it  is  likely  that  there  may  occur  instances  where  a  general  comment 
will  apply,  but  it  has  not  been  identified  specifically.  Hence, 
during  efforts  to  implement  changes,  all  GENERAL  comments  should  be 
kept  in  mind  by  those  responsible  for  actual  changes  to  the  code  to 
maximize  the  likelihood  that  all  relevant  occurrences  are 
corrected. 

The  comments  which  follow  use  the  designation  employed  on  the 
doctunentation  provided  to  identify  each  screen;  e.g.,  T0118. 
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(Comments  on  Level  I) 


TlOOl  G4.  Uses  the  word  "SOFTKEY"  without  defining  same 
beforehand  (i.e.,  "SOFTKEY"  is  computer  jargon,  apt  to  be 
unfamiliar  to  those  unfamiliar  with  computers) .  As  a  minimum,  the 
general  location  of  the  "SOFTKEYs"  should  be  cited  before  asking 
the  operators  to  employ  them  (as  is  done  on  the  succeeding  screen, 
T1002)  . 

T1003  G2.2.  EmBOLDen  the  word  "CONTINUE"  (or  reverse  video)  on 

the  last  line  to  be  consistent  with  the  practice  reflected 
elsewhere  within  the  screens  of  identifying  keys  to  be  pressed  by 
making  them  bold  (or  using  reverse  video)  .  Enclose  the  phrase 
with  double  asterisks. 

T1004  1.  Change  "AROUND"  to  "TO"  (Line  3). 

2.  EmBOLDen  "SELECT"  and  "K"  (or  use  reverse  video)  only. 

T1006,  TIOIO,  T1014  G3.  Delete  screen.  Incorporate  the  word 

"CORRECT"  in  place  of  the  term  "DIRECTIONS"  on  the  succeeding 
screen. 

TlOlO  G2.2. 

TlOll  The  last  line  of  text  ("YOUR  LAST  SELECTION:  x")  is 
confusing.  Delete  or  clarify. 

T1012  G4.  Because  this  is  the  first  reference  to  the  use  of  a 

"hardkey,"  suggest  the  last  line  as  follows:  THEN  PRESS  THE  ENT 
KEY  ON  THE  BOTTOM  LINE  OF  THE  NUMERIC  KEYPAD. 

T1015  G4.  In  consonance  with  the  orientation  of  the  preceding 

comment: 

1.  Indicate  the  location  of  the  CLR  key  after  the  word 
"OVER." 

2.  Insert  the  word  "SOFTKEY"  prior  to  "+/-"  and  indicate  its 
location  ("ON  THE  BOTTOM  LINE  OF  THE  NUMERIC  KEYPAD")  after  the 
word  "KEY." 

3.  As  in  the  preceding  comment,  incorporate  the  nature  and 
location  of  the  first-time-used  CONFIRM  softkey. 

4.  Additionally,  the  use  of  the  softkey  "CONFIRM"  to  indicate 
that  you  have  made  an  error  is  a  logical  inconsistency;  recommend 
that  the  "CANCEL"  softkey  be  employed  instead.  Throughout  the 
majority  of  the  ET  screens,  "CONFIRM"  is  employed  in  a  positive 
context — to  associate  it  here  with  the  need  to  correct  an  erroneous 
entry  is  inconsistent.  (If  CANCEL  cannot  be  used,  then  some  other 
term  should  be  generated  to  indicate  the  reed  to  revise  an 
erroneous  TOO  LARGE  or  TOO  SMALL  value.) 
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T1016  Place  "WHEN  FINISHED,  PRESS  CONTINUE  at  bottom  of  screen. 
(Note  the  emBOLDened  "CONTINUE.") 

T1017  G2. 

(Comments  on  Level  II) 

T0002  G6. 

T0003  1.  To  unambiguously  identify  previous  efforts,  the  software 

should  add  an  asterisk  or  otherwise  indicate  completion  of  each 
topic  at  the  time  it  is  finished — or,  conversely,  start  with  all 
items  with  asterisks  and  successively  delete  the  asterisk  as  has 
been  done  with  the  "SYSTEM  STARTUP  TRAINING"  screen  menu. 
(Individuals  were  observed  unintentionally  repeating  topics  during 
the  evaluation.) 

2.  G5.  (Re  "SYSTEM  STARTUP"  on  the  menu  or  "SYSTEM  STARTUP 

TRAINING"  on  the  screens) 

T0102  Present  wording  is  confusing  and  somewhat  contrary  to  logic, 
indicating  that  the  operator  is  to  "CONFIRM"  that  the  system  is 
"not  OK  "  as  the  basis  for  continuing.  Suggest  that  the  present 
screen  be  replaced  with  the  following: 

THE  NEXT  SCREEN  YOU  WILL  SEE  IS  THE  "POWER  UP  STATUS"  SCREEN.  TO 
PROCEED  WITH  TRAINING  ON  THE  PRESENT  TUTORIAL,  PRESS  CONTINUE 
REGARDLESS  OF  THE  STATUS  INDICATED  FOR  THE  ITEMS  SHOWN. 

Are  there  any  circumstances  that  you  would  not  want  the  operator 
to  continue?  As  it  reads  now,  there  is  no  indication  that  the 
operator  should  ever  discontinue  because  of  what  is  shown  on  the 
POWER  UP  STATUS  screen.  Is  this  true?  To  minimize  potential 
confusion,  it  is  recommended  that  the  text  be  revised  to  more 
clearly  indicate  what  is  occurring.  Is  it  necessary  to  introduce  a 
new  term  ("CONFIRM")  when  it  appears  that  what  is  being  asked  is 
simply  to  "CONTINUE"  (i.e.,  continue  in  spite  of  the  fact  that  all 
system  components  may  not  be  "OK")?  If  appropriate,  it  would  be 
more  straightforward  to  use  the  wording  indicated  above. 

T0104  G7.  DELETE. 

T0105  G2.1,  G2.2,  G3,  G4.  Confusing.  Multiple  instructions, 

apparently  applicable  to  a  screen  which  is  not  even  to  be  addressed 
in  the  Tutorial.  Clarification  and  revision  are  needed.  (See 


"NOTE" 

appearing  in  the  documentation  as  well 

TOllO 

G2.1, 

G2.2,  G3,  G5. 

T0112 

G2.1, 

G2.2. 

T0113 

G7. 

T0114 

G2.1, 

G2.2,  G3,  G5,  G6. 

T0116 

G2.1, 

G2.2,  G5,  G6. 

D-5 


T0117  G7. 

T0118  1.  G3 


2.  Confusion  between  the  use  of  the  term  "STARRED”  and  the 
actual  appearance  of  an  asterisk  was  evidenced  during  the 
assessment.  Change  to  the  use  of  the  term  ASTERISK  in  lieu  of 
STARRED. 


3.  Suggest  this  screen  be  broken  into  two  separate  screens 
as  follows: 

(TOllSa)  ON  THE  NEXT  TWO  SCREENS  IS  THE  "SETUP  AMD 
IMFORMATIOM"  MENU.  YOU  CAN  SWITCH  BACK  AND  FORTH  BETWEEN  THESE  TWO 
SCREENS  BY  PRESSING  THE  MEXTPAGE  SOFTKEY.  YOU  MUST  INITIALIZE  ALL 
OF  THE  ITEMS  WITH  ASTERISKS  (*)  ON  BOTH  PAGES  TO  START  THE  SYSTEM. 
AFTER  AN  ITEM  IS  SELECTED  AND  COMPLETED,  THE  ASTERISK  WILL 
DISAPPEAR  FROM  THE  MENU. 

**  PRESS  COMTIMUE  TO  PROCEED  ** 


(TOllSb)  DURING  ACTUAL  FIELD  OPERATIONS,  YOU  COULD  SELECT 
ITEMS  IN  ANY  ORDER;  HOWEVER,  TO  COMPLETE  THIS  TUTORIAL  YOU  SHOULD 
DO  THE  PAGES  IN  ORDER.  COMPLETE  ALL  ITEMS  WITH  AN  ASTERISK  ON  THE 
FIRST  PAGE  BEFORE  PROCEEDING  TO  THE  SECOND.  AFTER  COMPLETION  OF 
THE  FIRST  PAGE,  THE  SECOND  WILL  APPEAR  AUTOMATICALLY.  TO  BEGIN, 
SELECT  ANY  OF  THE  ITEMS  WITH  AN  ASTERISK  (*)  FROM  THE  NEXT  SCREEN. 
**  PRESS  CONTINUE  TO  PROCEED  ** 

4.  Inherent  in  the  preceding  cononent  is  the  need  to  include 
code  to  monitor  progress  and  determine  when  all  starred  items  have 
been  completed  and  automatically  switch  to  the  succeeding  screen 
when  all  items  with  an  asterisk  have  been  completed  on  the  first  of 
the  two  screens.  Recommend  further  that  upon  completion  of 
asterisked  items  from  both  pages,  the  Tutorial  automatically 
increment  to  a  separate  screen  announcing  that  they  have  completed 
"SYSTEM  STARTUP"  training,  and  directing  them  to  **  PRESS  COMTIMUE 
TO  PROCEED  **  (with  this  response  taking  them  back  to  T0003's  main 
menu) .  If  adopted,  this  approach  would  also  lead  to  the  deletion 
of  T0121  (and,  perhaps,  T0123) . 

T0122  1.  Delete  the  second  sentence  (lacks  context  and 

relevance) . 

2.  Change  the  "DO  NOT  ..."  to  an  affirmative  statement: 

RETURN  TO  THE  "SYSTEM  STARTUP  SCREEN"  AND  SELECT  AN  ASTERISKED 
ITEM. 

3.  Consistent  with  comment  2,  change  the  last  line  to: 

**  PRESS  COMTIMUE  TO  RETURN  TO  THE  "SYSTEM  STARTUP  SCREEN"  ** 

T0124  1.  This  screen  caused  significant  problems  with  many  of 

those  participating  in  the  evaluation.  More  information  is  needed 
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to  enable  the  HIP-naive  soldier  to  understand  what  is  meant  by  the 
first  two  sentences,  e.g.,  participants  were  unfamiliar  with  the 
concept  of,  and  the  need  to,  "INITIALIZE”  the  BCS  DATA  TRANSFER 
(As  reflected  in  the  MANPRINT  meeting  on  28  September,  such 
information  should  be  integrated  into  the  present  software, 
presented  in  a  separate  special  text  or  manual  to  accompany  the  HIP 
ET,  or  otherwise  made  available  to  the  trainer  and  the  trainee.) 

2.  G2.1,  G2.2,  G6.  Change  "SELECT  INITIALIZATION  FROM  ..." 

to:  ON  THE  NEXT  SCREEN,  MOVE  THE  CURSOR  TO  THE  INITIALIZATION' 
ITEM,  AND  PRESS  THE  SELECT  SOFTKEY. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0126  Same  as  comment  2  of  the  preceding  item. 

T0127  G7. 

T0128  1.  G3,  G5,  G6.  Change  first  sentence  to:  INITIALIZE  THE 

•TOT  RESPONSE  TIME'  ITEM  APPEARING  ON  THE  SUCCEEDING  "TIME-ON- 
TARGET  CREW  RESPONSE  TIME"  SCREEN.  Insert  "of  1.5"  after  the  words 
RESPONSE  TIME  in  the  second  sentence.  Substitute  the  following  for 
the  "PRESS  THE  ENTER  KEY  ...": 

ON  THE  NEXT  SCREEN,  ACCOMPLISH  THE  INITIALIZATION  BY  PRESSING  THE 
ENT  KEY  AT  THE  BOTTOM  OF  THE  NUMERIC  KEYPAD.  (NOTE  THAT  DURING 
ACTUAL  OPERATIONS,  YOU  WOULD  BE  ALLOWED  TO  ENTER  ANY  VALUE  BETWEEN 


.5  AND 

10.0 

.) 

T0129  Insert  hyphens  in 
G5:  Change  ENTER  to  ENT. 

the  screen  title:  TIME -ON-TARGET  ...  Re 

T0130 

G5. 

Change  ENTER 

to  ENT. 

T0131 

G7. 

T0132 

1. 

G3. 

2.  This  is  the  soldier's  first  contact  with  the  "NAVIGATION 
SUBSYSTEM  RESTART"  screen  and  menu;  recommend  revision  of  the 
initial  sentence  to  eliminate  the  notion  that  he  is  "restarting" 
something  he  has  not  encountered  previously. 

3 .  G2 . 2 . 

T0133  1.  G3. 

2.  Same  as  preceding  comment  2,  consider  deleting  the  first 
sentence  (RESTART  ...)  entirely;  it  is  redundant  with  the  preceding 
screen. 


3.  G2.1,  G2.2.  Replace  with: 

"PRESS  USE  ALL  ..."  with:  ON  THE  NEXT  SCREEN,  "NAVIGATION 
SUBSYSTEM  RESTART",  PRESS  THE  USE  ALL  SOFTKEY. 
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**  PRESS  CONTINUE  TO  PROCEED  ** 

T0135  Same  as  T0133. 

T0136  G7. 

T0137  1.  G3. 

2.  G2.1,  G2.2.  Replace  PRESS  YES  ...  with: 

ON  THE  NEXT  SCREEN,  "NAVIGATION  SUBSYSTEM  RESTART",  PRESS  THE  YES 
SOFTKEY. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0139  Same  as  for  T0137. 

T0140  G7. 

T0141  G2,  G3,  G5.  Replace  with: 

CORRECT.  NOW  INITIALIZE  THE  NEXT  SCREEN,  "PROPELLANT  TEMPERATURE". 
ALTHOUGH  A  RANGE  OF  TEMPERATURE  VALUES  FROM  -130  TO  +145  COULD  BE 
ENTERED  DURING  ACTUAL  OPERATIONS,  ENTER  THE  VALUE  SHOWN  ON  THE  NEXT 
SCREEN,  +70,  BY  PRESSING  THE  ENT  KEY  ON  THE  NUMERIC  KEYPAD. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0142  G2,  G5.  Add  the  location  of  the  ENTer  key  to  the  last  line 

and  enclose  in  double  parentheses: 

**  PRESS  ENT  ON  THE  NUMERIC  KEYPAD  ** 

T0143  1.  G5.  Change  ENTER  to  ENT. 

2 .  G2 . 2 . 

T0144  G7. 

T0145  1.  G3. 

2.  G2.1,  G5,  G6.  Replace  with: 

ON  THE  NEXT  SCREEN  "TUBE  LOAD  ELEVATION",  INITIALIZE  'LOAD 
ELEVATION.'  ALTHOUGH  DURING  ACTUAL  OPERATIONS,  ANY  VALUE  BETWEEN  - 
200  AND  +1333  COULD  BE  USED,  ASSUME  THE  LOAD  ELEVATION  OF  +300  IS 
CORRECT.  ON  THE  NEXT  SCREEN,  PRESS  THE  ENT  KEY  ON  THE  NUMERIC 
KEYPAD  TO  ENTER  THE  NUMBER  300. 

(Note  that  the  present  text  states  that  the  ENTER  key  is  on  the 
display  itself.) 

T0148  G7. 

T0149  G2,  G3,  G5,  G6.  (Comments  similar  in  nature  and  scope  to 

those  for  T0145.)  Recommend  inclusion  of  phrase  that  indicates  to 
the  operator  that  ALL  sector  settings  displayed  are  to  be  assumed 
to  be  correct. 
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T0151  1.  G2.2. 

2.  Change  PRESS  USE  ALL  ON  ...  to: 

PRESS  THE  SOFTKEY  USE  ALL  ON  THE  "DEFINE  SECTOR  OF  FIRE"  SCREEN. 
T0152  G7. 

T0153  1.  G3. 

2.  G2.1,  G5,  G6.  Replace  with: 

ON  THE  NEXT  SCREEN,  "SET  DATE  /  TIME  GROUP",  INITIALIZE  THE  DATE 
AND  TIME  USING  THE  KEYS  ON  THE  NUMERIC  KEYPAD.  USE  THE  CURRENT 

DATE  AND  TIME.  PRESS  THE  CLR  KEY  IF  _ 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0155  1.  G5.  Change  CLEAR  to  CLR 

2.  G2 

T0156  1.  G3 

2.  G2,  G6.  Replace  with: 

TO  SET  THE  CLOCK  TO  THE  DATE  AND  TIME  YOU  TYPED,  PRESS  THE  SOFTKEY 
MARK  ON  THE  NEXT  SCREEN. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0158  Change  the  second  sentence  to  read  as  follows: 

TO  SET  THE  CLOCK  TO  THE  DATE  AND  TIME  YOU  PREVIOUSLY  TYPED,  PRESS 
THE  SOFTKEY  MARK.  PRESS  MARK  ON  THE  NEXT  SCREEN. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0159  G7. 

T0160  1.  G3. 

2.  G2,  G5,  G6.  Replace  with: 

YOU  ARE  NOW  TO  INITIALIZE  THE  AMMO  INVENTORY.  ON  THE  NEXT  SCREEN, 
"INVENTORY  MANAGEMENT",  HIGHLIGHT  THE  'EDIT  CURRENT  AFCS'  OPTION  BY 
USING  THE  ARROW  KEYS,  THEN  SELECT  THAT  OPTION  BY  PRESSING  THE 
SELECT  SOFTKEY. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0161  &  T0165  1.  It  appears  that  the  "INVENTORY  MANAGEMENT" 

menu  involves  a  total  of  five  items  appearing  on  two  separate 
screens.  Is  this  true,  such  should  be  stated.  If  true,  can  the 
soldier  toggle  back  and  forth  between  the  two  screens  during  actual 
operations?  If  yes,  then  additional  explanation  is  needed.  (See 
additional  confusion  related  to  the  use  of  the  terminology  'AMMO 
INVENTORY'  in  the  comment  addressing  T0164.) 
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2.  The  ET  Testing  results  suggest  that  additional 
experience  with  the  ammunition-inventory-related  screens  would 
likely  prove  helpful  in  the  transition  between  Tutorial  Level  II 
and  a  Level  III  scenario. 

T0162  G2,  G5,  G6.  Beginning  with  the  second  sentence,  replace 

with: 

ON  THE  "INVENTORY  MANAGEMENT"  SCREEN,  HIGHLIGHT  THE  'EDIT  CURRENT 
APCS'  OPTION  BY  USING  THE  ARROW  KEYS,  THEN  SELECT  THAT  OPTION  BY 
PRESSING  THE  SELECT  SOFTKEY. 

**  PRESS  CONTINUE  TO  PROCEED  ** 


T0163  G7. 

T0164  1.  G2,  G3. 

2.  See  previous  comments  regarding  T0161  &  T0165.  The  next 
screen  introduces  three  additional  'INVENTORY  MANAGEMENT*  menu 
options,  and  each  is  referred  to  as  a  separate  INVENTORY .  However, 
the  soldier  is  told  to  assume  that  the  'AMMO  INVENTORY'  is  correct. 
There  is  no  actual  item  appearing  as  'AMMO  INVENTORY'  on  the  menu. 

3.  G2.2.  Change  the  last  two  lines  to  indicate  that  the 
FINISHED  key  is  to  be  pressed  to  terminate  the  present  screen  and 
proceed:  **  PRESS  THE  FINISHED  SOFTKEY  TO  INDICATE  THE  INVENTORY 
UPDATE  IS  COMPLETE  ** 

T0166  1.  G2,  G5,  G6. 

2.  Reconcile  with  previous  TOlSx-'series  comments 

T0201  G2.2,  G5.  Change  "Network  Access"  Display  to  "Network 

Access"  Screen.  The  use  of  "display"  may  confuse  computer 
illiterate  users. 

T0202  G2.2,  G3,  G5. 

1.  As  in  the  preceding  comment,  change  "display"  to  "screen." 

2.  To  unambiguously  identify  correct  current  values 

change  "verify  the  current  values"  to  "verify  the  values  shown." 
T0203  G2.2,  G3,  G4,  G5.  Present  wording  is  confusing  and  somewhat 

contrary  to  logic,  indicating  that  the  operator  is  to  press 
"Cancel"  to  discard  changes  made  and/or  to  verify  the  values. 
Suggest  that  "Cancel"  be  used  to  discard  incorrect  entries  and  "Use 
All"  or  "Select  All"  be  used  to  verify  values. 

"Cancel"  should  not  be  used  to  connote  nearly  opposite  meanings. 

T0204  and  T0206  G2.1,  G2.2,  G3,  G5.  As  indicated  previously,  the 

use  of  the  key  "Cancel"  to  exit  from  a  screen  without  making  any 
changes  is  somewhat  illogical.  Replace  "Cancel"  with  "Use  All"  or 
"Select  All." 
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T0207  and  T0208  G7.  Replace  with: 

CORRECT.  NOW  LEARN  HOW  TO  CHANGE  VALUES  ON  THE  "NET  ACCESS" 

SCREEN.  ON  THAT  SCREEN,  FIRST  USE  THE  ARROW  KEYS  TO  HIGHLIGHT  THE 
ITEM  TO  BE  CHANGED,  THEN  PRESS  THE  SELECT  SOFTKEY.  YOU  WILL  THEN 
GO  TO  ANOTHER  SCREEN  AND  SELECT  A  SPECIFIC  VALUE  OR  TYPE  IN  A  VALUE 
FROM  THE  KEYPAD. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0209  G2.1,  G2.2,  G3,  G6.  Replace  with: 

PRACTICE  BY  CHANGING  THE  'GUN  KEY  TIME  (SEC)*  ON  THE  MENU  APPEARING 
ON  THE  "MET  ACCESS"  SCREEN.  ON  THE  NEXT  SCREEN,  USE  THE  ARROW  KEYS 
TO  HIGHLIGHT  THIS  ITEM,  THEN  PRESS  THE  SELECT  SOFTKEY. 

T0211  G2.2,  G6.  Replace  lines  3  and  4  with: 

HIGHLIGHT  THE  'GUN  KEY  ITEM  (SEC)'  THEN  PRESS  THE  SELECT  SOFTKEY. 
T0212  G7. 

T0213  G2.1,  G2.2,  G3 .  Replace  with: 

CHANGE  THE  VALUE  TO  2 . 1  SECONDS,  ON  THE  NEXT  SCREEN.  USE  THE  ARROW 
KEYS  TO  HIGHLIGHT  2.1,  THEN  PRESS  THE  SELECT  SOFTKEY. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0215  G2.2.  Consistent  with  T0211  and  T0213,  replace  lines  4  and 

5  with: 

HIGHLIGHT  2.1  SECONDS  BY  USING  THE  ARROW  KEYS,  THEN  PRESS  THE 
SELECT  SOFTKEY. 

T0216  G7. 

T0217  G2.2,  G3,  G6,  G7.  Replace  with: 

CORRECT.  NOTE,  HOWEVER,  THAT  CHANGING  THE  VALUE  OF  A  SETTING  ON 
THE  "MET  ACCESS"  SCREEN  DOES  NOT  CHANGE  THE. . . 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0218  G2.2,  G3,  G6.  Insert  the  word  "SOFTKEY"  after  USE  ALL  and 

"SCREEN"  after  both  occurrences  of  "MET  ACCESS". 

T0219  G2.1,  G2.2,  G3.  Insert  "ON  THE  NEXT  SCREEN"  prior  to  PRESS 

USE  ALL;  insert  "SOFTKEY"  subsequent  to  that  phrase. 

T0221  G2.1,  G2.2.  Delete  "ON  THE  NETWORK  ACCESS  DISPLAY"  and 

replace  with: 

ON  THE  "MET  ACCESS"  SCREEN  PRESS  THE  USE  ALL  SOFTKEY. 

T0222  G7. 
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T0223  G2.1,  G2.2,  G3,  G6,  G7.  Replace  with: 

CORRECT!  NOW,  CHANGE  THE  'NET  ACCESS  DELAY  TIME  (SEC) . •  SELECT 
'NET  ACCESS  DELAY  TIME  (SEC) '  FROM  THE  MENU  ON  THE  NEXT  SCREEN. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0225  Gl,  G2.1,  G2.2.  Delete  lines  3  and  4  and  replace  with: 

ON  THE  NEXT  SCREEN,  USE  THE  ARROW  KEYS  TO  HIGHLIGHT  'MET  ACCESS 
DELAY  TIME  (SEC) '  THEN  PRESS  THE  SELECT  SOFTKEY. 

(Note:  As  in  other  circumstances,  the  sequence  employed  in  T0223 
and  T0225  infers  that  some  learning  has  occurred  by  this  point  in 
the  ET  sequence,  and  that  the  need  to  repeatedly  provide  detailed 
step-by-step  instructions  for  recurring  sequences  is  unnecessary. 
The  logic  at  this  point  is  that  the  individual  will  have  learned 
the  seqience  needed  to  "select"  a  menu  option  [re  T0223].  If  he 
hasn't,  the  error  correction  screen  [T0225]  repeats  the  more 
detailed  instructions  regarding  the  specific  steps  required  to  make 
the  correct  selection.  It  may  be  that  this  assumption  can  be 
employed  earlier  in  the  ET  sequence  than  is  presently  indicated.) 

T0226  G7. 

T0227  Gl,  G2.1,  G2.2,  G3 ,  G6,  G7.  Replace  ENTER  THE  VALUE  with 
TYPE  THE  VALUE. 

Consider  adding  a  screen  to  more  clearly  address  the  use  of 
decimals  (and  how  they  are  to  be  dealt  with  since  none  are  on  the 
keypad) ,  the  RETRY  SOFTKEY,  the  ENT  key,  and  the  apparent  fact  that 
(during  normal  operations??)  values  ranging  from  .5  to  49.5  may  be 
entered  (i.e.,  last  line  of  screen  T0228) . 

T0228  As  mentioned  in  the  preceding  paragraph,  delete  confusing 
reference  to  possible  time  ranges. 

T0229  G2.1,  G2.2,  G6.  Replace  lines  3  and  4  with: 

ON  THE  NEXT  SCREEN,  USE  THE  KEYPAD  TO  TYPE  THE  VALUE  8.5,  THEN 
PRESS  ENT. 

T0230  G7. 

T0231  Gl,  G2.1,  G2.2,  G3,  G6,  G7.  Replace  with: 

CORRECT!  NOW  CHANGE  THE  'BLOCK  MODE  SELECTION. '  ON  THE  NEXT 
SCREEN,  SELECT  THE  'BLOCK  MODE  SELECTION'  OPTION  FROM  THE  "MET 
ACCESS"  MENU. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0233  Gl,  G2.1,  G2.2,  G6.  Replace  lines  3  and  4  with: 

ON  THE  NEXT  SCREEN,  USE  THE  ARROW  KEYS  TO  HIGHLIGHT  'BLOCK  MODE 
SELECTION, '  THEN  PRESS  THE  SELECT  SOFTKEY. 

**  PRESS  CONTINUE  TO  PROCEED  ** 
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T0234  67 


T0235  62. 1,  62.2,  63,  64.  Insert  ON  THE  NEXT  SCREEN,  PRESS  THE 

YES  SOFTKEY  and  delete  ON  THE  NET  ACCESS  DISPLAY. 

Additional  explanation  regarding  the  "confirmation**  process  is 
needed.  Indicate  what  actual  question  is  asked,  where  the  question 
appears  on  the  screen,  and  the  availability  of  only  two  **values** 
which  **toggle**. 

Why  is  the  ** CONFIRMATION**  process  limited  to  the  **BAUD  RATE 
SELECTION**  menu  item?  Recommend  that  the  process  not  be  referred 
to  as  a  CONFIRM  procedure;  rather  describe  it  as  merely  a  different 
way  of  changing  the  entry  for  this  item.  INDICATE  may  be 
substituted  for  CONFIRM. 

T0237  62.1,  62.2,  66.  Consistent  with  previous  comment, 

substitute  INDICATE  for  CONFIRM.  Delete  NET  ACCESS  DISPLAY  from 
line  4  and  insert  BY  PRESSIN6  THE  YES  SOFTKEY  ON  THE  NEXT  SCREEN. 

T0238  67. 

T0239  62.2,  63,  66.  Substitute  SCREEN  for  DISPLAY. 

T0240  62.2,  66. 

T0242  62.2. 

T0301  62.2,  66.  Sub|titute  SCREEN  for  DISPLAY. 

T0302  62.2,  63,  66.  Substitute  SCREEN  for  DISPLAY. 

T0303  62.2,  63,  66. 

T0304  62.1,  62.2,  63,  66.  Replace  lines  2  and  3  with: 

FROM  THE  NEXT  SCREEN,  SELECT  * BROADCAST  ADRS. * 

T0306  62.2,  66.  Do  not  highlight  SELECT. 

T0307  67. 

T0308  61,  62.1,  62.2,  63,  66,  67.  Replace  with: 

CORRECT 1  NOW,  CHANCE  THE  VALUE  OF  THE  * BROADCAST  ADRS,*  ON  THE 
NEXT  SCREEN  TO  J.  USE  THE  ARROW  KEYS  TO  HICHLICHT  J,  THEN  PRESS 
THE  SELECT  SOFTKEY.  SELECT  J  FROM  THE  **MET  ADDRESS**  MENU. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0310  61,  62.1,  62.2,  66.  Replace  lines  2-4  with: 

CHANCE  THE  VALUE  OF  THE  * BROADCAST  ADRS*  ON  THE  NEXT  SCREEN  TO  J. 
USE  THE  ARROW  KEYS  TO  HICHLICHT  J,  THEN  PRESS  THE  SELECT  SOFTKEY. 
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T0311  G7. 


T0312  62.2,  63,  66,  67.  Replace  line  1  with: 

CORRECT!  THE  OTHER  ADDRESS  SETTINGS,  THOSE  ENDING  IN  ADRS, . . . 
Replace  'BROADCAST  ADDRESS'  with  'BROADCAST  ADRS.' 

T0313  62.2,  63,  66.  Substitute  SCREEN  for  DISPLAY. 

T0401  62.2,  66.  Substitute  SCREEN  for  DISPLAY. 

Additionally,  information  regarding  the  time  requirements 
should  be  presented  earlier  in  Level  II,  along  with  a 
recommendation  that  it  be  addressed  first — see  T0415,  T0416. 

T0401.5  62.2.  Consider  repositioning  this  screen  to  the  end  of 

the  section,  as  a  "wrap-up"  note. 

T0402  62.2,  63,  66.  Replace  with: 

THE  "NAVIGATION  SUBSYSTEM  RESTART"  SCREEN  SHOWS  THE  POSITION  VALUES 
CURRENTLY  STORED.  YOU  CAN  VERIFY  AND  ACCEPT  THE  CURRENT  VALUES, 
ENTER  NEW  VALUES,  OR  GET  THE  POSITION  FROM  THE  NAV,  THE  BCS,  OR  THE 
LAST  MOVE  ORDERS. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

Also,  the  meaning  of  NAV  needs  to  be  clarified. 

Recommend  introducing  the  "NAVIGATION  SUBSYSTEM  RESTART" 
screen  earlier  (e.g.,  after  T0401)  to  allow  the  soldier  to  have  a 
more  concrete  appreciation  of  its  contents.  Presently,  he  has  to 
deal  with  it  in  the  abstract  through  four  screens  before  seeing  it. 
If  this  recommendation  is  adopted,  the  last  lines  of  T0401  could 
be: 

THE  "NAVIGATION  SUBSYSTEM  RESTART"  SCREEN  IS  SHOWN  NEXT. 

FAMILIARIZE  YOURSELF  WITH  IT.  WHEN  FINISHED,  PRESS  CONTINUE  TO 
LEARN  MORE  ABOUT  IT. 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0403  62.2,  63.  A  better  description  of  the  broader  perspective 

is  warranted  prior  to  T0403  to  more  adequately  indicate  that  a 
principal  objective  of  this  section  is  to  enter  all  updated  data 
(or  accept  that  stored)  needed  to  begin  the  alignment  sequence. 
Revise  the  present  screen  to  reflect  the  fact  that  USB  ALL  is  a 
command  which  accepts  (and  stores)  all  data  shown  as  well  as 
starting  the  alignment  process  (e.g-,,  PRESSING  USB  ALL  ACCEPTS  ALL 
VALUES  SHOWN  AND  STARTS  THE  ALIGNMINT  PROCESS) . 

T0404  61,  62.1,  62.2,  63,  66.  Replace  with: 

ON  THE  NEXT  SCREEN,  "NAVIGATION  SUBSYSTEM  RESTART,"  SELECT 
'ALTITUDE' . 
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T0406  Gl,  G2.1,  G2.2,  G6.  Change  "SELECT  ALTITUDE  ..."  TO;  ON 
THE  NEXT  SCREEN,  "NAVIGATION  SUBSYSTEM  RESTART,"  USE  THE  ARROW  KEYS 
TO  HIGHLIGHT  'ALTITUDE',  THEN  PRESS  THE  SELECT  SOFTKEY. 

**  PRESS  CONTINUE  TO  PROCEED  ** 


T0407  G7. 

T0408  Gl,  G2.1,  G2.2,  G3,  G6,  G7. 

1.  The  word  TYPE  has  been  used  previously  to  indicate 
numeric  information  is  to  be  entered  from  the  keypad.  Substitute 
TYPE  for  ENTER. 

2.  Indicate  that,  as  shown  on  the  "NAVIGATION  SUBSYSTEM 
RESTART"  screen  during  normal  operations  values  from  -999  to  +9999 
may  be  entered. 

3.  The  system  should  be  consistent  in  addressing  the  use  of 
-  and  +  signs,  particularly  the  latter  (e.g.,  see  T0142  in  the 
"propellent  temperature"  screen) . 

T0409  G5. 

1.  The  directions  to  TYPE  ALTITUDE  -999  to  9999  suggests 
that  another  number  is  to  be  entered  instead  of  the  400  value 
indicated  (see  Comment  2  above) . 

2.  Replace  PRESS  ENTER  with  THEN  PRESS  ENT. 

T0410  G2.1,  G6.  Replace  lines  3  and  4  with: 

AT  THE  NEXT  SCREEN,  USE  THE  KEYPAD  TO  TYPE  400,  THEN  PRESS  THE  ENT 
KEY. 

T0411  G7. 

T0412  G2.2,  G3,  G5,  G6,  G7. 

1.  Use  bold  letters  (or  reverse  video)  for  'EASTING, ' 
'ALTITUDE,'  'GRID  ZONE,'  'FAST  ALN  AZ, '  and  'SPHEROID.' 

2.  Does  the  necessity  of  being  approximately  correct  apply 
to  only  'SPHEROID'  or  to  any  of  the  position-related  values?  If  it 
is  applicable  to  other  position-related  values,  so  state. 

T0413  G2.2,  G3,  G6.  Information  provided  on  this  screen  is 

ambiguous  and  confusing. 

1.  The  statement  'GET  NAV  POSITION'  GETS  THE... FROM  THE 
NAVIGATION  SUBSYSTEM  is  unclear.  The  soldier  is  already  working  in 
the  NAVIGATION  SUBSYSTEM.  Does  this  statement  refer  to  information 
previously  stored  in  the  navigation  subsystem?  If  so.  Insert 
"INFORMATION  PREVIOUSLY  STORED  IN"  after  "THE"  at  the  end  of  the 
third  line. 
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2.  The  last  sentence  is  also  confusing.  Does  "ENTERED” 
mean  the  same  as  "FIRST  BEGUN"  when  initially  beginning  the  process 
of  entering  values  in  the  "NAVIGATION  SUBSYSTEM  RESTART?" 

T0414  G2.2,  G3,  G5,  G6.  Use  bold  letters  (or  reverse  video)  for 

'GET  LAST  MOVE  ORD, '  'EASTING,'  'NORTHING,'  'ALTITUDE,'  'GRID 
ZONE,'  and  'GET  BCS  RPRT  P08. ' 

T0415  G2.2,  G3,  G5,  G6.  Replace  lines  1  and  2  with: 

THE  ALIGN  MODE  CAN  BE  EITHER  NORMAL  ('ALIGN  MODE  NORMAL')  OR  FAST 
('FAST  ALIGN  AZ ' ) .  YOU  MAY  CHOOSE  EITHER. 

T0416  G2.2,  G3,  G6.  Time-relevant  statement  should  be  placed 

closer  to  preceding  time-relevant  information.  Revise  in  the 
following  manner: 

A  STATUS  BOX  ON  THE  TOP  LINE  OF  THE  DISPLAY  SHOWS  THE  APPROXIMATE 
TIME  NEEDED  TO  COMPLETE  THE  ALIGNMENT.  PRESSING  THE  USE  ALL  KEY 
STARTS  THE  ALIGNMENT  PROCESS.  ONCE  STARTED,  IT  CONTINUES 
AUTOMATICALLY.  YOU  CAN  LEAVE  THE  "NAVIGATION  RESTART  TRAINING" 
SECTION  AND  BEGIN  WORK  ON  OTHER  SECTIONS  OF  THE  SYSTEM  AS  IT 
CONTINUES . 

**  PRESS  CONTINUE  TO  PROCEED  ** 

T0417  G2.2,  G3,  G6. 

T0420  G2.2,  G3 . 
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APPENDIX  E 

LIST  OF  ACRONYMS  AND  ABBREVIATIONS 
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ARI 


; 


AFCS 

AS  ARC 

ASVAB 

COS 

ET 

GT 

HIP 

lOTE 

PM 

ROC 

SGT 

SSG 

SFC 

SHE 


U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences 

Automatic  Fire  Control  System 

Army  System  Acquisition  Review  Council 

Armed  Services  Vocational  Aptitude  Battery 

Chief  of  Section 

Embedded  Trainer 

General  Technical 

Howitzer  Improvement  Package 

Initial  Operational  Test  and  Evaluation 

Product  Manager 

Required  Operational  Capability 

Sergeant 

Staff  Sergeant 

Sergeant  First  Class 

Subject  Matter  Expert 
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